430 research outputs found
Cost Preserving Bisimulations for Probabilistic Automata
Probabilistic automata constitute a versatile and elegant model for
concurrent probabilistic systems. They are equipped with a compositional theory
supporting abstraction, enabled by weak probabilistic bisimulation serving as
the reference notion for summarising the effect of abstraction. This paper
considers probabilistic automata augmented with costs. It extends the notions
of weak transitions in probabilistic automata in such a way that the costs
incurred along a weak transition are captured. This gives rise to
cost-preserving and cost-bounding variations of weak probabilistic
bisimilarity, for which we establish compositionality properties with respect
to parallel composition. Furthermore, polynomial-time decision algorithms are
proposed, that can be effectively used to compute reward-bounding abstractions
of Markov decision processes in a compositional manner
Process algebra for performance evaluation
This paper surveys the theoretical developments in the field of stochastic process algebras, process algebras where action occurrences may be subject to a delay that is determined by a random variable. A huge class of resource-sharing systems – like large-scale computers, client–server architectures, networks – can accurately be described using such stochastic specification formalisms. The main emphasis of this paper is the treatment of operational semantics, notions of equivalence, and (sound and complete) axiomatisations of these equivalences for different types of Markovian process algebras, where delays are governed by exponential distributions. Starting from a simple actionless algebra for describing time-homogeneous continuous-time Markov chains, we consider the integration of actions and random delays both as a single entity (like in known Markovian process algebras like TIPP, PEPA and EMPA) and as separate entities (like in the timed process algebras timed CSP and TCCS). In total we consider four related calculi and investigate their relationship to existing Markovian process algebras. We also briefly indicate how one can profit from the separation of time and actions when incorporating more general, non-Markovian distributions
Logic and model checking for hidden Markov models
The branching-time temporal logic PCTL* has been introduced to specify quantitative properties over probability systems, such as discrete-time Markov chains. Until now, however, no logics have been defined to specify properties over hidden Markov models (HMMs). In HMMs the states are hidden, and the hidden processes produce a sequence of observations. In this paper we extend the logic PCTL* to POCTL*. With our logic one can state properties such as "there is at least a 90 percent probability that the model produces a given sequence of observations" over HMMs. Subsequently, we give model checking algorithms for POCTL* over HMMs
A probabilistic extension of UML statecharts: specification and verification
This paper is the extended technical report that corresponds to a published paper [14]. This paper introduces means to specify system randomness within UML statecharts, and to verify probabilistic temporal properties over such enhanced statecharts which we call probabilistic UML statecharts. To achieve this, we develop a general recipe to extend a statechart semantics with discrete probability distributions, resulting in Markov decision processes as semantic models. We apply this recipe to the requirements-level UML semantics of [8]. Properties of interest for probabilistic statecharts are expressed in PCTL, a probabilistic variant of CTL for processes that exhibit both non-determinism and probabilities. Verification is performed using the model checker Prism. A model checking example shows the feasibility of the suggested approach
Bounding the Equilibrium Distribution of Markov Population Models
Arguing about the equilibrium distribution of continuous-time Markov chains
can be vital for showing properties about the underlying systems. For example
in biological systems, bistability of a chemical reaction network can hint at
its function as a biological switch. Unfortunately, the state space of these
systems is infinite in most cases, preventing the use of traditional steady
state solution techniques. In this paper we develop a new approach to tackle
this problem by first retrieving geometric bounds enclosing a major part of the
steady state probability mass, followed by a more detailed analysis revealing
state-wise bounds.Comment: 4 page
Multi-Objective Approaches to Markov Decision Processes with Uncertain Transition Parameters
Markov decision processes (MDPs) are a popular model for performance analysis
and optimization of stochastic systems. The parameters of stochastic behavior
of MDPs are estimates from empirical observations of a system; their values are
not known precisely. Different types of MDPs with uncertain, imprecise or
bounded transition rates or probabilities and rewards exist in the literature.
Commonly, analysis of models with uncertainties amounts to searching for the
most robust policy which means that the goal is to generate a policy with the
greatest lower bound on performance (or, symmetrically, the lowest upper bound
on costs). However, hedging against an unlikely worst case may lead to losses
in other situations. In general, one is interested in policies that behave well
in all situations which results in a multi-objective view on decision making.
In this paper, we consider policies for the expected discounted reward
measure of MDPs with uncertain parameters. In particular, the approach is
defined for bounded-parameter MDPs (BMDPs) [8]. In this setting the worst, best
and average case performances of a policy are analyzed simultaneously, which
yields a multi-scenario multi-objective optimization problem. The paper
presents and evaluates approaches to compute the pure Pareto optimal policies
in the value vector space.Comment: 9 pages, 5 figures, preprint for VALUETOOLS 201
Deciding Probabilistic Automata Weak Bisimulation in Polynomial Time
Deciding in an efficient way weak probabilistic bisimulation in the context
of Probabilistic Automata is an open problem for about a decade. In this work
we close this problem by proposing a procedure that checks in polynomial time
the existence of a weak combined transition satisfying the step condition of
the bisimulation. We also present several extensions of weak combined
transitions, such as hyper-transitions and the new concepts of allowed weak
combined and hyper-transitions and of equivalence matching, that turn out to be
verifiable in polynomial time as well. These results set the ground for the
development of more effective compositional analysis algorithms for
probabilistic systems.Comment: Polished version with a more complete running example and typo fixe
- …